Home › HARMONY 2011 Exit Survey

HARMONY 2011 Exit Survey

Following HARMONY 2011, the organizers conducted an anonymous electronic survey to assess what worked and what didn't work. We wanted to learn lessons that could be applied to future meetings. This page presents the results of the exit survey. The content below has been slightly edited to remove some details that might have allowed easy identification of who wrote the comment, in order to better preserve anonymity.

Question #1: If you had to pick one thing that you wish had been different about the meeting and could be improved, what would it be?

  1. Bigger fonts for nametags! This is a perennial issue.
  2. Pepare the detailed schedule more in advance to better allow planing.
  3. Location was very 'happening' and all, but it all seemed to be happening to my wallet. It was atrociously expensive. Also, having people have to go find their own lunch kind of broke down the community feel, and took much more time than if something had been provided to the room, or such.
  4. The scheduling. The organisation of each day seemed a bit haphazard and largely relied on people being in the right room at the right time to hear about particular discussions being arranged. I think this was partly due to being the first HARMONY and needing to work on better intercommunication between the different projects participating in the meeting - an issue discussed extensively on the final day of the meeting so possibly already being addressed for future meetings.
  5. Too much presentations?
  6. Lunch be provided.
  7. Increase the hackiness of it all. The atmosphere was more akin to a conference than a hackathon.
  8. Posting presentations on the wiki BEFORE the presentation.
  9. The tutorials were a waste of time. Most of the APIs have getter/setters, and I felt the presenters were trying to fill up a lot of that time as best as they could to say more.
  10. I would have liked more tutorials for "beginners". At the end, a section of the website "co.mbine.org" for "training" has been proposed, and I think a space for training should be a substantial part of the meetings.
  11. Scheduling of break-out sessions was slightly confusing, and it was hard to schedule a session without overlap with another important session. The cbio people bravely attempted to create a rough outline for a schedule. But perhaps a different system could be attempted next time; if it would be possible to have a system where you can see which rooms are free, schedule meetings and join up for meetings, community members could self-organize the schedule without a lot of input from the organizers. Even better if it was possible to schedule meetings a week in advance. This could be as simple as a public wiki, or perhaps something more advanced. But surveymonkey doesn't work well for this purpose, because you can't see what is reserved and you can't see what other people chose until too late. In short, my suggestion is to make the scheduling process completely self-organizing, before and during the conference.
  12. A better use of technology to support more efficient communication. There were too many times where people asked 'what should I do now?', 'where should I go now?' - you really want to get answers to these questions as quickly as possible. If these questions are not answered quickly then people start wasting time (checking email, leaving the meeting to go somewhere else). Being in a location that doesn't have so many attractions outside the meeting room would help as well, but ideally, the organization and smoothness of communication should suffice. For example, there could be a website that enables people to link up into subgroups working on specific goals and can sort out the scheduling to reduce overlap. One thing we could explore is allowing people to edit a social network such that it updates in real time and the visualization is constantly available on a website and projected on a screen during the meeting. People could rate their affinity (0-10 point scale) to any number of topics/goals, or suggest new topics, and then clusters in the network would be natural subgroups. The majority of this work ideally would be done before the meeting, such that it can be refined during the meeting. There also needs to be a very clear link from the social network to the physical working space so people know where to go - this reduces the case where person X in location A wants to work on task Y, but doesn't know that a bunch of people are currently working on that task in location B. Also, there should be more breakout spaces (these don't have to be separate rooms), but you also don't want to have a case where a bunch of people want to work on something, but then they can't find a space to work.
  13. It would be nice to know more about the agenda at an earlier time
  14. Despite being a hackathon and not a presentation-based workshop, a minimal structure should be provided for each day, including advised plenary times. The self-generation of breakout worked well. But people not attending those were a little lost in particular on Friday. This encouraged skipping.
  15. Would it be possible, without disrupting the spirit of hackathon, to organize the time somewhat more, or somehow give participants more of a sense of things to be accomplished during the meeting?
  16. Introductions - it was generally a small enough group that it would have been nice to say who we are and why we were there (or to share that info via a web page if it were obtained during registration).
  17. A bit earlier organization, less spontaneous. Less talks in general, more time for individual discussions.

Question #2: If you had to pick one thing that you think worked well about the meeting and you think should be done the same way in the future, what would it be?

  1. The relatively free formatting that allows hacking on code to happen.
  2. I loved the idea of a 'quiet room' in addition to the break-out rooms.
  3. The time for detailed discussions.
  4. I think the concept of the meeting worked well. the act of getting different groups to the same location to discuss their collaborative working was great. It changed the dynamics of how a hackathon works, but I think thats just something that we need to live with as we are covering more groups and their conventions in working.
  5. Accommodating everyone in the same hotel nice and close to the meeting venue.
  6. There was always a good selection of things to eat and drink available in (or next to) the room.
  7. The availability of rooms for breakout sessions.
  8. Gave coverage to all data formats equally.
  9. a. Many rooms to go to for personal discussions. b. Possibility to do videoconferencing from one of the extra rooms.
  10. Generally, the focus on a particular standard each day was good.
  11. I liked that there was both time for sbgn/sbml/biopax meetings as well as joint/interop. The balance was perfect
  12. Ample time and room to work on new ideas that weren't planned before the meeting.
  13. The facilities and support were excellent - that is not something you want to think about during a meeting - it should just 'work'.
  14. It was very nice that the organizers were able to provide free registration and accommodation.
  15. The availability of multiple rooms, including a "quiet" one worked well. We could actually even have had an additional one.
  16. It looked like the right people were there (not to say other combinations couldn't also have worked well, but this one clearly promoted a lot of interactions that mostly sounded really productive).
  17. The brief tutorials giving overviews of the various projects being discussed were interesting and helpful!
  18. The coffee breaks, the food provided (cake, coffee, cookies). The location was also great.

Question #3: What did you achieve during this meeting? (It can be more than one thing.)

  1. Consensus!
  2. Met people who will be important going forward with my work - in terms of initial discussions on how to collaborate, and 'social' interactions..
  3. Became a lot more familiar with SBGN and BioPAX than I had been previously; started looking at how models I usually work with might be expressed in SBGN-PD and realising that they might be better in SBGN-ER; learnt about libSBML and jSBML and PAXTools; learnt a whole lot about OWL vs RDF/XML, SPARQL, and related technologies from the semantic web community; got really confused with where my own work fits in with related efforts underway under the COMBINE umbrella and then started to sort things out following discussions.
  4. The main thing was a better understanding of what needs to be done in terms of validation.
  5. I had a great overview of all the activities and I identified a couple of areas where I could work
  6. Learn more about each effort; meet key individuals; be able to provide feedback; create new collaborative opportunities.
  7. Made good progress on AF-support in LibSBGN Made good progress on extra validation rules for LibSBGN Started on SBML plug-in for PathVisio based on jSBML helped out with SBGN-ML support in Kegg converter and Paxtools Created Wikipathways QC-script using BioPAX validator webservice.
  8. It was very useful to see the growing interest in cooperation and convergence between the formats. I also found it very useful to learn about flux balance analysis and cell perturbation theory.
  9. Closure on several SBGN issues that had been going on for a long time. Start of a collaboration between Bio2RDF and MIRIAM URI infrastructure.
  10. a. Progress on development / refinement of BioPAX b. Better sense of the relationship between pathway and modeling resources c. Better appreciation of the importance of "falt", highly participatory social structures for this community

Question #4: How were the achievements facilitated by your presence at HARMONY?

  1. The fact that everyone (or at least enough people) was there. The fact that we could make presentations. Everyone's attitude.
  2. I think the social aspect is quite important, so its a good way to get to know people and their work.
  3. The tutorials; presentations from various key members of the different communities participating in HARMONY; discussions with various other participants; having the time out from my regular working schedule to focus on specific aspects of the topics being presented and discussed at the meeting.
  4. The ability to discuss and with faster back and forth with a wider group interested in validation.
  5. It is virtually impossible to get in touch with such an amount of information in just a week. I got in touch with creators, designers, developers. Through email, that would have taken months.
  6. I don't think it would have happened otherwise.
  7. For example, I was able to help [...] with SBGN-ML support in [...] by setting up the starting point for the converter within an hour. If we had to do this online, it would have taken 10 times as long. Similarly, [...] showed me to set up access to the BioPAX validator webservice in a very short time, [...] checked our AF diagrams and [...] fixed bugs in my schematron rules. All of this would have been possible over the internet, but it would have been 10 times slower.
  8. I helped show the potential of convergence between BioPAX and SBML. I managed to avoid getting into a fight with [...] and instead we found various agreements :)
  9. Getting people arguing face to face is the only way to understand each other enough to solve things. Meeting [...] was very important and point to the importance of having people peripherally involved present at the meetings, and not just the hardcore afficionados.
  10. Getting a large, representative (I hope) group of interested people in one room improved the quality / relevance of the discussion tremendously.

Question #5: Tell us about the good, the bad, and the ugly, in your opinion, about HARMONY 2011.

  1. The only slight difficulty was the room scheduling, but there was always somewhere we could go, so it worked out in the end.
  2. Bad: a.) Costly location. b.) Organisation was not optimal (hotel rooms unbooked, coffee missing, no breakfasts/lunches, no evening sightseeing arranged) c.) Break out session organisation was abysmal - I put my name on the board to 4 potential breakouts, but was not tapped up at all. I was also surprised by the number of people who actually put their name to anything (was much lower than it should have been). It was also clear that many people, for whatever reason, felt it was below them to even approach that board. d.) Did I mention coffee? At one point I could not decide if it was 'late' for the morning session, or if it had come for 'post lunch'. e.) Hotel internet - too slow/randomness if you got logged in at all f.) Hotel - lifts were crap Good: Morning talks. Good idea to see whats happening in the field/related fields Location - it was nice (also bad - see above) Hotel - mostly good choice snacks/coffee (when there) internet.
  3. The mix of snacks available was great, as was the coffee and water when they were available. The venue in general was fantastic, and the range of meeting rooms available was also great - a big thanks to the local organisers for sorting it all out! The range of tutorials was good, although there were some gaps - limited, I imagine, by fitting everything into one day. Maybe future meetings could allow for people to submit requests or proposals for specific tutorials they would like to attend/give (kind-of happened this year, but with a predetermined range of possible topics). Could cover more ground by having some centralised presentations and then break out into different tutorials focussed on specific topics/tools/software.
  4. Compared to the previous joint meetings, this is definitely a more productive one. Someone did a good job setting up the schedule.
  5. I didn't like the restaurant choice for the group dinner. The presentations lasted much longer than they should have in some cases. For the afternoon session, it might be good to have like 90 minute slots. Hotel internet was bad. The only time I really felt like there are no more discussions, no more presentations, all that is left is to actually hack was Friday afternoon. There should be more of that, perhaps the whole day Friday.
  6. The good is that it's a small meeting, and you get to meet just about everybody. I liked that there was goodwill, and the desire to collaborate, particularly along the inter-group boundaries. i think it was very productive!
  7. Great hackathon!
  8. For me, this was one of the most successfull meetings ever in terms of gaining insight into the promising direction the community is going and helping to make progress. Overall, the decision processes need to be more transparent.
  9. The good: Nadia. The organization was not as slick as we could have hoped, in particular compared to past years. For instance, the transfer of information through the website was a bit suboptimal. But without Nadia, this meeting would have been a disaster! The bad: Over the years, we have been spoilt by a few individuals who took care of things and provided the sacred flame. It seems that they are a bit demotivated, or maybe suffer from a dilution effect. The ugly: all the people who came only for a day or less and basically got subsidized vacations in New York.
  10. Tutorials - definitely useful in principle but could be improved by more organization of presentations, perhaps handouts or links in advance, to help relative beginners to get started better. Tutorials would have benefitted from more eplicit examples / problem-solving. These are all statements that, I think, are true about any teaching, and all call for extra, hard work by the teachers and students so how far it's realistically possible to go in this direction isn't clear.


Comments about the length of the meeting

  1. Length was fine. To be honest it could have been longer, but that would really be pushing it.
  2. For a meeting to cover all these standards, I think the length is just right.
  3. It would be better if it were longer, but I doubt that is possible.
  4. I only came for the last 3 days, so this worked out fine for me.
  5. I imagine some people find a week too long, so perhaps it's a good idea to make the first and last day less important. Perhaps first morning and last afternoon should be completely devoted to break-out sessions.
  6. I wish I could have stayed the entire week. More than a week would have been too much, except perhaps for one extra day.

COMBINE